DocLLM is a lightweight extension to traditional LLMs for reasoning over visual documents, considering both textual semantics and spatial layout. It avoids expensive image encoders and focuses on bounding box information. It outperforms SotA LLMs on 14 out of 16 datasets across all tasks and generalizes well to previously unseen datasets.
Keywords:
Keywords:
